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ABSTRACT 


This thesis is a case study that compares actual 
costs to estimated costs for a State-of-the-Art (SOA) 
extension. The Advanced Nuclear Gamma-Ray Spectrometer 
(ANGAS) program initiated by the Defense Advanced 
Research Project Office (DARPA) in conjunction with the 
Research and Development Division of Lockheed Missiles 
and Space Company (LMSC), MC icmimae SUD)JECt of this 
Case Sstuady. 

meme reol oe Gentiades; the original description of 


tmicoeiioloovVweexbLencionw; the methods used at Lockheed 


COmedese lope SOA e€xtension cost estimates; Specific 
Paclore stan melped and edit toe Gl acouica Le Cos t 
estimating and Petit icant “varianees in cost and 


bechimerwoacical progress. 
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I. INTRODUCTION 


A. BACKGROUND 

The development of new weapon systems for the 
Department of Defense (DOD) frequently requires State 
of-the-Art (SOA) extensions. The, difficultveasot 
estimating development and early production costs for 
new systems when SOA extensions are involved often 
leads to cost overruns. In fact, contractors scocnm 
exceed project manager’s estimates for new projects 60 
percent of the time. [REF. 1: p. 50-51. | 

Prior research has not solved the problem of 
accurately estimating SOA extension costs. These 
development and production cost overruns make it 
difficult for DOD to accurately compare existing weapon 
technology with undeveloped technology. 

Professor Willis R. Greer, Jr., Naval Postgraduate 
School, Monterey, California, in conjunction with ene 
Naval Sea Systems Command (NAVSEA), is attempting to 
develop more reliable methodology for estimating the 
cost of extensions as applied to electronic systems. 
This thesis provides data to support that effort 
through the case study of an SOA extension. 


B. OBJECTIVES 

This thesis details the Advanced Nuclear Gamma-Ray 
Spectrometer (ANGAS) experiment program initiated by 
the Defense Advanced Research Projects Office fia 
conjunction with the Research and Development Division 
of Lockheed Missiles and Space Company, Inc. Tie 
objectives are outlined below: 


1s. Identify the original description of the 
technology extenci on, 


One Document estimated development costs, 


oe teeeripe mene methods wsed to devehop the cost 
estimakes , 


Compare actual cost data to estimated costs, 


aye Analyze and measure any Significant variances 
in cost Or PEOsSress dead technological 
evelopment, 

OF ieeatlimyeaspeecrrtec factors that help or hinder 


accurate cost estimating. 


Toe Eo RAR Cn OUBST IONS 

Primamgyenresearch @uestion: 

HOw sQOmmmec SEIMNalLeGmaecvelopmenct and production costs 
compare with actual costs for the ANGAS project? 
Subsidiary skesearch Questions: 

se weat “methods ATO Pe cuMraues were = used to 


ecorrmace devell@pment and production costs? 


Ae How was technological progress measured and 
evaluated? 

ci Wat factors helped or hindered accurate cost 
estimating and accomplishment of stated SOA 


extension goals? 


D. SCOPE LIMITATIONS AND ASSUMPTIONS 

[hice thesis is a case study of the ANGAS project. 
Ine .Ewnary LtLaskeis tG obtain and analyze estimated and 
actumameme@st data, document cost estimation procedures 
for the ANGAS project and to evaluate generic cost 
estimating procedures sie cl by Lockheed for SOA 
projects. The technological requirements of the ANGAS 
project are studied to measure the accomplishment of 
stated ANGAS technological goals. 

TheweiGAS project, an ongoing effort at Lockheed, 
is designed to produce one end unit to fly aboard a 
sate I lpigitee Tins eee cost data Anes 1 Nation 
pee@eeaUnesmperesented in this paper are more applicable 
Lome wetepicmt COsts rather than to production costs. 


tii weammioorbaie because Many SOA Extensions, by their 


nature, must be implemented through a substantial 
development effort. At the outset of many SOA 
development efforts, the end unit is more of an 
abstract idea than a detailed production specification. 

In the 1970’s, system acquisitions programs were 


usually focused on specific technical approaches at 
the time of their initiation. 


Today a pee ram is initiated after (competent 
authority, the Program Decision Aurora. aD eiaees a 
specific formally . stated mission need, ased on 
Mission Area Analysis, submitted within the first 


Program OO ee es Memorandum_in which program funds 
are requested. |REF. 2 : p. 1-7. 4 


Ee. METHODOLOGY 
The methodology used in this research is case 
study. Research trips to Lockheed included visits with 


the ANGAS Contract administrator, principe 
investicacor, senior cost estimator and several 
development engineers. These Lockheed personnel are 


located at facilities in Sunnyvale and Palo Alto, 
California. Lockheed personnel provided estimated 
costs, the associated cost estimation methods, actual 
costs incurred to date, a summary of progress, 
technological breakthroughs and specific factors that 
have helped and hindered the SOA extension effort. 

The NAVPRO office, located in Sunnyvale, was 
visited numerous'7 times to identify a suitable contract 
for study and to obtain relevant contract and cost 
data. 

The ANGAS program manager was also interviewed to 
determine what the government hoped to achieve from the 
ANGAS project and how technological progress of that 
project is measured. 

Background and research data were obtained through 


the Naval Postgraduate School library and its research 


services. Automated searches were conducted through 
the NPS library’s DOD, NON-DOD and Defense Technical 
Information Center’s files. Information obtained 
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Gerowene visits Sto Lockheed during the case study and 
related research material is contained in this thesis. 


F. SUMMARY OF FINDINGS 
Major findings of this thesis are summarized below: 
plier There were no significant variances between 


Zowiiote td aaee Lua Casts tO Gate. Spending 
was at a slightly lower rate than anticipated. 


Ca DbisemmMany research and development projects 
UI bemaaministered by a “level.of effort 
eomtract wemeasurement and analysis of 


Lee MmMoOhagLecal developmenu 1s the primary gauge 
GemecrleCe@mmvyeness anmem@propgress. 


si pOemextensilons;, achieved by numerous smaller 
Serec  Dieuiccus tat incrementally extend the 
peeeeeicauce the risk of beth costs overruns and 
failure to achieve technrotopical goals. 

4. peweonune lweosts@are the most .sipnificant cost 
fener in tre Gevelopment plase of SOA 
espenslons. 

De. Incremental funding of a DOD SOA project 
Dm oc omleverave TO Che —eovernment that can 
Pemuce scant OVErrins and achieve desired 
Poo on ee Beals bub venas to ine@mease the 
lolmvieOlmetime required for the development 
et Tout. 

Ou UU Tet y eavallable parametric models, used in 
private PEL 7, COU O De are) eee 


O 
Bpevernment Use with proper historical data 
application and case-by-case analysis. 


Ge OR GANTZATION OF STUDY 

Chapter II provides a background of the ANGAS 
project, an overview of cost estimation procedures used 
for ANGAS and a summary of cost estimation procedures 
and models used at Lockheed. 

Ptioamecr lil cetalils ~the methodology used in this 
nie ctomaleampresenit data. 

Chaemer IV analyzes the data in Cnapuer Ii} 
Coucuocwmemuenwith the stated objectives. 

Chapter V_ states the conclusions drawn from this 


DOM Cx temamon case study. 


II. BACKGROUND 


A. PROJECT OVERVIEW 

ee ate Se Advanced Re See eo imlectis Agency 0]. 

fabrica Pere the ANGAS flight design, | test, aye 

performance ra a One coe flight demonstration in 

Space. [RE ox =| 

The ANGAS project is designed to develop a gamma- 
ray spectrometer that will be carried into orbit by 
Sabi ll ive: The spectrometer is designed to pasiver, 
detect radiation in space through an array of germanium 
sensors. The project will be completed when the 
designed spectrometer has been flown in space and the 
specified scientific results have been supplied to the 
government. The specific requirements of the ANGAS 
project are provided in the Statements of Work listed 
in Appendix B. 

The ANGAS system required development of a large 
multi-element array of new, n-type, segmented germanium 
sensors. The design concepts represent a research and 
development effort which has never been performed 
before. This SOA extension in gamma-ray spectrometers 


incorporates the following new technologies: 


hs Imaging collimator, 
oe Cooled, radiation resistant n-type germanium 
sensors, 


Ab COm meddle nm Ce ms dene idle, 

4. Electronics subsystem which enables rapid 
onboard data analysis and microprocessor 
Eontrom, 

Oc: Solid cryogen cooler. 

The ANGAS program is being developed by Lockheed 
Space Sciences Laboratory which is part of the Lockheed 


Research and Development Division, the principle 
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research organization of the Lockheed Missiles and 
Space Corporation. 

The government’s program manager for ANGAS is LTCOL 
George P. Lasche, USA. He is in the Geophysical 
Sciences division at the Defense Advanced Research 
Projects Office, Arlington, VA. He described the ANGAS 
program goals as: 

Seneitivitys and capability to recard and passively 
detect radiation seri sacs” through a = process o 
imaging with high energy resolution. 

A wide variety of potential DOD uses for ANGAS 
include its use as a means to verify arms treaty 
compliance from space. 


B. OVERVIEW OF COST ESTIMATION PROCEDURES USED FOR 
ANGAS CONTRACT PROPOSALS 
UNS ereateoerlelis method used by Lockheed Space 
Selences Laboratory to determine the cost estimates for 
ANGAS contract proposals is a bottom-up, engineering 
estimate. To date approximately $4.96 million has been 
Pumeaen theetwo increments, in a “level of effort" type 
Chime wee lie MeOroyecteaq tCOtal funding for the entire 
ANGAS project will be approximately 17 million dollars. 
Is, George Nakano, Lockheed’s praine mp re 
investigator (project director / technical supervisor) 
for ANGAS explained that the ANGAS cost estimate: 
is a bottom-up estimate that is developed based _ on 
pactecmaperience of costs gincurred developing similar 
satellite borne systems. 
ie bottom-up estimating methodology used at 
Lockheed provides detailed functional and cost element 
estimates provided by the lowest competent level in the 


organization. The cost estimates developed for each 


1 Information was Omea ined by telephone 
conversation on September 3, 1987. 

2 Information was obtained in an interview at Palo 
AMSLG, een on september 4, 19387. 


phase of the ANGAS contract were based on historical 
cost data, the requirements of the Work Breakdown 
Structure (WBS) and the Statement of Work (SOW) items 
listed in the Requests for Proposal (RFP). 

Lockheed’s contract pricing proposal for ANGAS 
contained the following summarized items. [REF. 3 : p. 
2.2-2.3] 


le. The overhead, general and administrative rates 
are applied as a percent times direct labor 
dollars. The rates were forecasts based on 


historical data and current projections. 

Oe Material requirements are identified, listed 
and priced by component or subsystem. These 
direct purchase items range from firm prices to 
estimates based on the accuracy that program 
requirements were defined. 

ok Procurement burden is applied as a percent of 
the total estimated costs for material, 


purchased serices and subcontract and outside 


production costs. The rates are based on 
historical data and current projections. 
4. Cost of money is an element of facilities 


capital. The rates are applied to the direct 
labor dollars associated with annual overhead 
and general andadministrative rates and to the 
base amount associated with the annual 
procurement burden. The cost of money rate used 
was 10.375 percent. 

-. Air fare rates are averages of recent Lockheed 
ticket costs fOrR tourist class to the 
designated cities. Per diem rates are based on 
historical data and current projections. 

Ore Direc labor hours, the most signifieant 
component of the ANGAS cost estimate, are based 


on a technical evaluation developed froma 


Ib 


detailed analysis of the program requirements. 
The basic labor rates are current average rates 
for direct labor pools, by skill categories 
that are used in the program. These rates are 
escalated by approved factors when appropriate 
to -alVowsiror future labor rate increases during 
Pie wot Ooi am. 

Wil aeocuoc dnGmamp  PoG@uratveseawere audited by the 
Defense Contract Audit Agency and negotiated with the 
CooniZamteO@overnment activity. Spee! 1 Cummecests, rates 
anc Aucmmresults wall be discussed in Chapters III and 
sli ae 

Labor hour and cost estimates were based on 
See veuemesaucellite iilitont programs involving similar 
but much smallér gamma-ray sensor systems developed by 
Poenkhiceas the Defense Advanced Research Projects Agency 
301 program although only one-forth the size of ANGAS 
and less technically complex was used in developing the 
ANGAS cost estimate. 


C. SUMMARY OF COST ESTIMATION PROCEDURES AND MODELS 
Usb by LOCKHEED 
tie iiise Estimating Systems Description (ESD) 
Manual provides company-wide guidance for estimating 
Peg uiene mem ts for government or Dieeeine Customer 
cOontraccs. 
ite =erscC ESD Manual states: 
OV eta lr respomcaod li ty LOMeariewmLvioc estimating 
definition, development, control and compliance is 
vested in the Vice-President, Finance. The Division 
tS PweceoUnbants report to the Vice-President 
Finance and are responsible for ensuring compliance 
Wittieemwes GCocument. | REF. 4 : p L.3] 
The following summary of cost estimation procedures 
and models is provided from information obtained from a 
Prescemeervon on SOM pricing by Mr. Ted Castro, Manager 
MiSsCeeaeemating Systems, Mr. Donald H. Palmby, Manager 


imMeCeCost — Modeling and Analysis, and Mr. Ken Peeler, 
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Chief Industrial Accountant LMSC Astronautics Division 
and the LMSC ESD Manual. 
1. SOA Proposal Estimating Process 

Figure 2-1, which follows this description of 
Lockheed’s contract proposal estimating procedure 
provides a flow diagram of the process.[REF. 4 : p. 
ee meee 2) 

The cost kick-off meeting establishes a 
schedule of deadlines for completion of cost inputs. 
Cost inputs and bases are generated for materials, 
direct Wander and eiuner Giewecteca- tc. 

The Federal Acquisition Requirements’ (FAR) 
directs DOD contractors te: 

provide. a consolidated price summary of individual 

material quantities included in various tasks, orders 

or contract line items being proposed and the basi 

PorpPricing, (vendo quotes, invoice prices, etre... 
Material requirements are broken down into 

three categories. 

Ke Subcontract estimates: obtained through formal, 
detailed RFPs or through estimation if 
sufficient time is not available to identify a 
subcontractor and obtain a price quote before 
Lockheed must respond to the government 
proposal. Lockheed is required to establish the 
reasonableness of any estimate through an 
independent cost analysis or technical basis. 

gee Material estimates: includes prices for 
standard commercial items, raw materials, 
purchased parts and material usage costs. These 
material items are priced through historical 
data, supplier quotations and estimations. 

3. Intra-Lockheed Work Transfer (IWT) Items: these 
material items are made at Lockheed. FAR 


requires that these items’ be transferred at 
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cost with an explanation of the pricing method 

is 6 Gly. 

Once all material requirement costs. are 
Consolmuaatea they are reviewed and additional analysis 
Poe lisacit tcatlom= lo provided 1i needed or required by 
FAR. A consolidated price summary of all material 
Pedulrenenrs 1Ssmemaen generated. Lockheed’s final action 
Tor material costs is the application of material 
Sa@abattommratescei£ JUStitied for a particular contract 
OF COntract. phase. 

Direcrslaber cost estimates must; 

Ie otae “Nourse, rates’ and coat bs 

: ¥ 
eBEImbtes “CRESS S82 "FAR 18"Soa-sy7™ SP Pass FoF 

[Pewech Labemmeectimaltes@are Usually expressed by 
Lockheed in labor hours, identified directly to 
mesecarch, design, production or other project 
objective. To justify direct labor estimates, the 
requirements of an SOA extension RFP are broken down 
into tasks consistent with SOW requirements' or WBS 
element. The two basic methods used to provide a basis 
of estimate (BOE) for hours quoted are: 

ilies Feestorbecal or similar to data: this is the 
preferred BOE for quoting direct labor hours. 
Similar programs are used to identify tasks 
that are equivalent to tasks to be performed in 
the proposed contract. Adjustments or applied 
factors are often used to calibrate historical 
fata uStit heatten. Or adjustments or factors 
is required by FAR. 

a Engineering estimates: used when no historical 
or similar to data is available. Tasks are 
broken down to the lowest level of detail. 

Use of estimating standards is a direct labor 
estimating method which relates production costs to 


Seecimeteoucharacteristics Of “a product, such as weight, 


ie 


size or composition. 
Cost ee standards must be applied 
consistently based on verimied uniformly correlated 
cate COlmae a in the same unit of necIS ue Rae ae 
A learning curve, which prediets the (exten 
which experience in performing a task will decrease the 
labor time needed to perform that task, is then applied 
by Lockheed to all labor estimates. The LMSC labor cost 
accumulation system is the data base used to develop 
learning curves. Curve selection and slope definition 


depend on a number of factors, including: 


des how the data has been accumulated (by lot 9 or 
Un ieee) 
on whether the item is a follow-on to an existing 


production run or an estimate based on a 

Similar iten, 

Ons Lie <2 production break, change in design 
technology or factory method has occurred, and 
4. customer specification of the methodology, 

[REF owe: -p 2mol ) 

Once total labor hour estimates are compiled 
and reviewed Lockheed direct labor ~hour) ratecean- 
applied to the labor hours. The estimation of “other 
direct costs" and BOE is the final major cost category 
that must be derived for an SOA extension price quote. 
Other Vda reerrcos tswinec iude. 

(APC}/pooled work brders, monitored line parts Apt, 
reloca Vonmerexpensces anc foreign field service 
bonlisess| REM. 6 3 Dp. 2-62 

Other direct costs are usually based on 
leis tO i ie cul eone Gmiect estimating procedures. Aww 
estimates for materials, direct labor and other direct 
costs form the initial cost (preoposatewareh ais now 
reviewed and approved or disapproved. If approved 
management price review and negotiation and sometimes 
best and final offer steps follow. 
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Zo Bottom - Up Cost Estimating Method 
Lockheed uses a multiple estimating approach. 
When detailed task definition is available a bottom - 


up / engineering estimate is prepared by the cognizant 


division(s). These bottom - up estimates provide: 
ee Heoneaergree ONneaccuUuracy and traceability, 
Ces eeu breakdowns to the lowest levels of the 
oe Based on historical data. 
The primary disadvantages of a bottom - up cost 


estimate for an SOA extension effort are: 


i les Dependence on detailed design specifications 
Which do not usually exist during early phases 
of SOA extension projects, 


On Expensive and time consuming, 
SA Cannot be used as the sole costing method 
4. Trade - off analysis is not easily evaluated. 


Although many SOA extensions do not initially 
provide detailed design specifications, bottom - up 
cost estimating is sometimes still used by Lockheed. 
Painless “are Nipaseca on) the experience gained by 
individual engineers and technicians from developing 
systems or components that used existing SOA 
technology. Many of the SOA extension contracts awarded 
to Lockheed are an extension of technologies and 
systems developed previously by Lockheed. Bottom - up 
SOA extension cost estimates are reconciled with 
experience and computer cost models. 

Various computer models are used in SOA cost 
estimating by Lockheed to establish "price targets", 
provide "sanity checks" and reconcile other cost 
estimations. 

3. Parametric Cost Models 
Lockheed uses the RCA PRICE Hardware (H) 


parametric cost model early in the development or 
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concept exploration stages of an SOA extension. 


The PRICE model contains thousands of mathematical 
equations relating the input variables to cost. Each 
specific set of input Be cet ee defines the 
hardware for cost modeling. The resultant cost output 
is determined from the mathematical equations alone. 
Bae not perform the function of a table look- 
up model. 


PRICE has been. designed to_estimate costs with a 
minimal amount_of hardware information. This feature 
makes it a legitimate tool for cost estimation of 
programs in concépt stage of development,since the 
modél uses its internally generated values for anM 
missing input variables in order toestimate _ cost. O 
course,it is always preferable for the PRICE user to 
supply the inputs, when their values are known. In 
nis weg tag statistical uncertainty is reduced. 
sak 


Lockheed applies the PRICE H model when 4key 
representative system characteristics are Known." 
PRG ee also identifies cost drivers’ and 
quickly and economically evaluates SOA extension trade- 
off and schedule analysis. 
The major disadvantages of the PRICE H model as 
noted by LMSC’s senior cost estimating managers are: 


le Not based on actual history, 
es Algorithms need to be adjusted for different 
projects, divisions and acquisition 


environments. 
PRICE H relies on input of a subassembly or 
higher level unit’s active electronic weieht sang 
mechanical/structural weight as the most critical cost 
and schedule factors. The accuracy of the PRICE output 
is directly related to the accuracy of the weights 
in pute 
Input characteristics of the PRICE 4H model 
which enhances its value as an SOA extension cost 
estimation model are: 
ee Inputs consider the lead time necessary for 


-up, parts procurement and redesign. This is 
significant for most SOA extensions. 


3 Information obtained from LMSC SOA cost 
presentation, September 24, 1987, Sunnyvale, CA. 
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H ie vid 


Manager. 


itewcestrees, Ol Smew. desiten and complexity of 
development engineering tasks is included. 


The operational environment in which the system 
is to be used is evaluated in cost estimates. 
SOA extensions such as the ANGAS project are 
designed to be used in a new operational 
environment, outside the earth’s atmosphere. 


de Anan oon ingmeomnolexitye of the 
electronics within the system or component is 
evaluated by the model. New electronics design 
is ee a Significant factor of SOA extension 
costs. 


The model also evaluates technological 
improvement, yeas of technology and 
hechmoOmogo teal Getay ser lags. These inputs are 


GihtenwoUAnextension cost drivers. 
Itiustration 2-1 is a Sample output of a PRICE 
provided by LMSC Cost Modeling and Analysis 


The output has been broken down into six 


sections. The following general output descriptions are 
summarized from the PRICE Executive Handbook [REF. 7 


Oe Ole 


Relevant SOA extension output data is also 


heen toned. 


Section 


>eet Lon 


1; identifies time of the run, the name of 
lee ei fedepateny si) information about the 
item, the recurring unit production cost 
and monthly Boer on RPoceSreetillus. 2-1 
identifies the item as a pyro-kit, to be 
produced in a quantity of one. Mics hg abe 
weight was input at 7.96 pounds, the unit 
volume was .26 cubic feet and mode _ two 
designates the item as a mechanical item 
with no electronic elements. The quantity 
/ next higher See designator is a 
one which determines the number of units 
required to be integrated and tested at 
the next higher assémbiy level. 


2: lists program costs in engineering and 
pa ae ae subtotals. mimstration 2-1 
indicates. on y production costs of 08 
dollars. Development costs are blank, in- 
dicating that no development costs are 
required. oy SOAwextenctons require sig- 
nificant development costs. The ability of 
PRICE Heeewoutpucs development and 
production costs separately_is useful in 
analysis of technological alternatives. 
lWelus. 2-1 @eists Empinecring costs of 
$504. These costs are composed of 
drafting costs of $46., design costs of 
$250., project management costs of $148. 
and data costs of $60. Manufacturing 
costs total $505. These costs are 
eae Of predueelomecosts Of $453. and 
tools and test equipment costs of $52. 
Total engineering and manufacturing costs 


IU, 


Section 3: 


Section 4: 


Section 5: 


Section 6: 


equal $1008. 


provides OPE PU useful for analysis of the 
inputs with known standards. Density 

engineering changes and mean time between 
failures values in Illus. 2-1 are followed 


by an asterisk. This indicates that these 
input values were not known and calculated 
by the model. Many inputs _ required to 
estimate the costs of an SOA extension are 
not available during the early phases of 
development. The ability of the PRICE H 
model to derive output and then provide 
missing input is ideal for SOA extensions 
durimpgeear x phases. Illus. 2-1 shows 
mechanical design factors that were input 
for the run. The manufacturing complexity 
of 10.42 is an on eaeca ay derived value 
that Ree Ses the product’s 
producibility which is a function of 
material type, finished density and 
fabrication methods. The mechanical 
integration level of .7 represents the 
level of mechanical integration and test. 
This factor describes the level of effort 
required for the integration of mechanical 
equipment in the next higher assembly. The 
platform input of 2.2 designates the 


intended operating environment. An input 
of 2.0 or more represents unmanned or 
manned space operations. The year 


of 
technology, 19385, has been calculated by 
the program. The pes 120 a factor of 
1.0 is a multiplier used to deviate the 
mean time between failure value which has 
been calculated by the model. 


outputs development and/or production 
schedule information. [llus. 2-1 indicates 
that the dates for completion of the first 
item and overall completion were not known 
before the run. The model provides valuable 
schedule information which is often 
difficult to determine for SOA extensions. 


prowess additional, in~depth cost output. 
he model again PON Ee previously unknown 
information which would be valuabl 

for SOA extension esti 


e 
mat oe First unit 
production costs are Spee 
1 


iL 
17, amortized 
unit costs are $1008.3 the production 
tooling cost factor is 1.0 and the unit 
Rees curve is 86.4 percent. 

The production cost multiplier _ of 1.20 is a 
ea factor used to include mock-ups 
for general and administrative and fee or 
profit in the production ceocteoutputs. 


one of the most useful outputs of the 

PRICE H model is the cost uncertainty 
measure. Illus. 2-1 provides a cost range 
of 954 to 1046 dollars. SOA extension 
development and production costs inherently 
contain a high level of pega t Oty 

due to the technical innovation an 
engineering complexity. 
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4. tnembocktweed SITAR Costekstimating Model 
Lockheed’s STAR cost estimation model combines 


the characteristics of a similar-to model and an 
associated model. A Similar-to methodology uses 
historical cost data from tasks or equipment similar to 
the equipment under evaluation. A high degree of 
accuracy and cost breakdowns to the subsystem or box 
level (WBS level 3 or 4) is provided. 

The associated program methodology uses total 
annual costs from technically representative projects 
that have been adjusted by economic and complexity 
factvors: 

The basis of the STAR model is the detailed 
historical database that has been developed over a 20 
year period. This data base represents) actual 
production and development cost experience, rather than 
SUCCESS Oriented, or tailored data. 

The STAR model is made up of three distinct 
estimating models with separate data bases for each. A 
flight hardware cost estimator, a ground hardware cost 
estimator and a software cost estimator. The individual 
models use an algorithm designed forGbts particular 
line of business. The software cost estimator works 
With a software size estimator that calculates lines of 
code by functional application. The three models can be 
used in an integrated manner to derive a total system 
CoS ts. 

Minimum inputs include: program hardware _ / 
software lists, a master schedule and the year dollars 
required. Critical SOA extension estimating factors 
such as: design complexity, escalation of costs, and 
quantity are incorporated in the STAR model. 

The primary advantage of the STAR model 
compared to the PRICE H model is that output is derived 


2k 


from actual or historical costs. STAR also allows 
updating of costs as a project progresses and tradeoff 
and "what if analysis". 

The STAR model contains a program loader that 
pulls the required type of data from the data base. The 
FILGEN pulls analogous cost items from the data base. 
Each item on the program hardware list is coded with a 
box status, box type and engineering status. 

Box status codes are either: make (m), buy (b) 
or government furnished equipment (g). 

Box type codes are either: development (qd), 
qualification (qjmer produciiem super 

Engineering status codes are either: new (n), 
modified (m), existing (e) or production (p). 

Each item on the program hardware list is also 
assigned input values for: quality, management effort, 
design engineering require, systems engineering 
required, software engineering effort required,the 
manufacturing complexity, test requirements and product 
assurance. 

The model now makes a cost run, the Lockheed 
labor rates are the default labor rate values. Appendix 
c4 provides sample output for a subsystem program 
hardware list. 

Final output includes; totale nours doe era. 
value for the labor hours, material costs in dollars 
and total dollar value for each item on the subsystem 
hardware list. The final total price is obtained from 
analogous, historical cost data. If any of seven input 
values were not initially known the total cost is 


normalized and reallocated back to the missing boxes. 


4 APPENDIX 


CW ise not par Of the. unl iced 
distribution of this _ thesis. Toda valent within DOD may 
obtain a copy of... APPENDIX from Professor Willis R. 
Greer, eioeons Naval: Postgraduate School, Monterey, 


California. 
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The STAR model is an ideal tool for Lockheed to 
use to estimate SOA extension costs. The data base is 
retrievable by line item or box level, which enables 
composite data from even classified projects’ to be 
included as unclassified useable data. 

If a particular line item on the hardware parts 
Requirement list e's Seeingeerto be subcontracted or 
purchased directly, the program allows these Known 
costs to be entered. Otherwise a data base is selected 
which matches he functional and characteristic 
qualities of the item. The default learning curve 
Vauem ia JO Percent. 

The output of the STAR model is designed to aid 
SOA cost analysis. Output can be formatted to provide: 
development/non-recurring costs and production / 
recurring costs. Appendix C provides sample output in 
this format andan example of output in WBS reference 
number sequence. 

The accumate, Uniform measurement and 
GerprelactiOnm Of historical data into box level oor 
component factor enables extremely precise SOA 
extension cost estimations by the STAR model once parts 


requirements are well defined. 
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Figure 2-1 (Continued ) 


as 


TLLUSIT RATIONS 2-1 


Ta 
INPUT FILENAME: EX cae opie eo 16:56 
GLOBAL FILENAME: LMSCGLO1 
PY ROpeek iT 
PRODUCTION QUANTITY 1 
UNIT WEIGHT 7.96 MODE 
UNIT VOLUME 0.26 QUANTITY/NHA iL 
UNIT PRODUCTION COST 452.67 
MONTHLY PRODUCTION RATE 0.06 
Te 
Gn COST DEVELOPMENT PRODUCTION TOTAL COST 
ENGINEERING 
Drafting - 46 46 
Design - 250 aot) 
systems - = — 
Project Mgmt = 148 148 
ata = 60 60 
Subtotal(ENG) - 504 504 
MANUFACTURING 
Production - 453 453 
PE Ormeeyne as = = 
Tool-Test resi - De Ore 
Subtotal (MFG ) 505 505 
TOTAL COST 1008 1008 
Lia 
DESIGN FACTORS MECHANICAL PRODUCT DESCRIPTORS 
Weight 7.960 Platform 2.2 
Density 20. 0s Yeateeon . leche 1985 
Mise. Complexiey 10.422 Reliability Fac. la 
Engineering Change OU 2a MTBF(Field) 21971* 
Integration Leve 0.700 
Tee 
SCHEDULE START Poko Gl FINISH 
Production Jan 85 (19) Jul 86* (0) Jul 86* (19) 
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ILLUSTRATION 2-1 (Continued) 


ve 

SUPPLEMENTAL INFORMATION 
Economic Base PSI oom sand Process Factors 
Escalation 0.00 ,rProduction Tooling 1.00* 
T-1 Cos 452.17*Rate ee 0 
AMOr= Untescost i0USm@e0*Price Info Factor 0.950 
Prod Cost Mult 1.205Unvs Learning Curve 0.864 

We 

COST RANGES DEVELOPMENT PRODUCTION TOTAL COST 
From = 954 954 
Center - 1008 1008 
To ~ 1046 1046 
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III. METHODOLOGY AND DATA 


A. INTRODUCTION 

This chapter describes the methoedotocy so tae 
thesis and presents data {hat was collecteq sau, te en. 
S tue Ine Me toed ores, section is divided into two 
subsections, data collection and data analysis. 

Data "tne wudes: estimated costs submitted by 
Lockheed in the ANGAS contract proposals; Det emcac 
Contracts Audit Agency (DCAA) audits of the ANGAS 
CONtTrTAaAct Pop osas.. costs questioned in thesere 
audits; final negotiated costs with justifications; 
actual costs ~incurred at ditferent  (stasceswo ea 
contract; statement of work items; a istine orem en 
MNpkestewes. 


B. METHOUCEOGY 
1. DatawColtect ron 

The methodology used in this research is case 
study. An information search was conidducted to obDGaum 
background data. Automated searches were conducted 
through the Naval Postgraduate Schoo library seo 
NON-DOD and Defense Technical Information “Cemten 
files. The following literature provides information 
on SOA extensions, dévellopment and )preduct lonmeoee 
estimation issues and a background that will enhance 
undersGandin se f the issues in this case _ study. 
Literature that is pertinent to data analysis 
methodology will be presented later in this chapter. 

Captain Helmut Wik 2.5 =Ssehece emt oAL. in ‘ees 
master’s thesis [REF. 1:p. 1-91], studied the methods, 
techniques and objectives used in estimating costs for 
exploratory development projects by Air Force Research 


and Development Laboratories. He also attenmican ae 
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McMeel ceevOors ewhich “contribute to the variance 
between pigey Cc Naneacer: s cost estimates and 
contractor’s proposal costs. Captain H. Scheel 
surveyed 45 government project managers to determine 
cocm cStLIMating — techniques. Important conclusions of 
ees “study include: 


1. 77.8 percent of the program managers’ used 
historical data for general cost estimating. 

om Se-cmebercent Of % the prorpram Mose! & used 
historical data and expérience/judgment to 
estimate labor costs and manpower requirements. 

3. Low cost estimates were sometimes made because 
of limited funds available for ae specific 
Prowyect.. 

4. The major variance factor attributable to the 


DitteersS WasSwe@rOns im estimation as a result of 
Neca (oT PP eee on of the Statement of Work 


i Contractors can sometimes propose a different 
level of effort than specified n an SOA 
extension due to technical insight and 


capabilities known only by the contractor. 


om Establishment of a Sopa ae Gaspard system with 
standardized software would provide a data bank 
that would enable more accurate cost estimation 
analysis within the government. 


ve. Most re vyeceu managers do Not Meaccurately 
estimate costs for new projects. The majority 
underestimate by 30 percen or less, although 


ome underestimate by as much as 290 perce. 
fer. oewesa, so,.41, 51, 67; 92-96 


The ANGAS project, like most SOA extension 
efforts, requires a long research and development (R&D) 


period. To accurately estimate and evaluate proposed 
contractor costs an understanding of long-range 
corporate R&D planning is useful. Ward C. Lowe framed 


the process into seven significant steps: 


ir | as Glear E as possible the basic 
Hoeninologileal, objectives which are of primary 
mpeeres t. These may be stated broadly at the 


beginning but must eventually be broken down 
Mico Darticwular areas. 


Cus Identify the goals_ toward which the company is 
working, which tdeally are set forth in the 
corporate and the research objectives. 


Be. ea pee DLDg with the preceding two factors 
seriously and imaginatively consider ail | 
possible results which may be achieved if the 
research efforts are successful. 


oo 


4. Rank the SO ee eo capabilities of the 
research efforts as determined in the eee elas 
Lone in terms of their potential contribution 
to the achievement of specific goals. The 
ranking should achieve the corporate or the 
research objectives. 
OG Outline the principle technological Sten Which 
are required to achieve the hypothetical 
research results listed in the preceding step. 
such an outline should uncover the more 
significant gaps in existing knowledge. 
6. Select the small number of high-value research 
results, as determined previously, which have 
reasonably well-specified steps and a minimum 
number of knowledge gaps. ive these results 
further examination and consideration for full- 
scale research efforts. 
ee Remain alert to any developments that could 
significantly change the value ratings used in 
the preceding steps. When research 
breakthroughs fill in ae knowledge gaps, 
639 process should be repeated. TREF- 8:p. 62- 
The above steps are also useful for government 
program managers when defining the technological goals 
of SOA extension projects and drafting the SOW. 
Accomplishment of SOW items during the development 
phase of an SOA extension contract may be the primary 
measurement tool for evaluating technological progress. 

The bottom-up, parametric and similar-to-cost 
estimating models described in Chapter II are the most 
widely used cost estimating methods. References 4, 6, 
and 7 provide information for further study. 

It was time consuming and difficult to identify 
a company willing to cooperate in a case study which 
offered an SOA extension project useful for analysis. 
Several Marketing and Research and Development 
Directors were contacted by telephone to determine 
interest in and suitability for study. 


Companies initially contacted included GTE 


[inte National Semi-Conductor Ines Fairchild ine... 
Advanced Microcircuit Devices Inc.; Cypress Electronics 
1G ote: Acrian Inc.;: Hewlett Packard Lime. : Harris 
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Microwave Inc; ime ll SCemp- itiemeawm Litton Industries 
ie mand baton semiconductors Inc. 

Company responses ranged from "not interested" 
to enthusiastic requests for additional information 
about the research effort. Companies expressing an 
Prbcerestweinm “aiding the research effort were then 
evaluated on their ability to provide viable projects 
Or ee sluUay . Three other Naval Postgraduate School 
students had selected SOA extension case studies for 
their theses. All of us worked as a team to identify 
appropriate firms. These efforts resulted in the 
Serco ronwor. GlE Ine. and Litton Industries,einc., as 
candidates for study by two other graduate students. 

LCDR. Fred Voellm, SC, USN, of the wNavy 


Procurement Office located at LMSC, was contacted for 


assistance in identifying a government contract 
administered by his office for an SOA extension that 
could be studied. LCDR. Voellm produced a list of 


contracts that were considered high technology and 
potential candidates for study. 

Individual contracts were evaluated on the 
degree of new technologies and SOA extension, the 
meoievapility Of initial and actual cost data and the 
stage of completion of the contract. 

The ANGAS experiment program initiated by the 
Defense Advanced Resear on Pe Onecins Office in 
PMc mlen With stne Researehn and Development Division 
of LMSC was selected because it combined numerous SOA 
becom@loeleal Extenstfonss santo a design that was unique 
and required operation in space. Selection of the 
iow omen teect, Which is still in early research and 
design “stages, provided an Sep OhtuMmmny LO StUdCY 
development costs and how technological progress is 
evaluated: 


Sab 


Mr. Don Darse mn, LMSC’s ANGAS contract 
administrator, was visited numerous times to obtain 
background and cost data. The ANGAS "Weekly Financial 
Status Report" and "Work in Process Ledger by Work 
Order" provided actual current costs and spending 
rates. A summary of this data will be presented in the 
data section Yer ) ents” chapter. Mr. Dorsett also 
discussed Ede significant technological and SOA 
extension characteristics of the ANGAS project. These 
are listed in Chapter II. 

Leo: George rare Lasche, USA, Geophysical 
Sciences Division, Defense Advanced Research Projects 
Office , Caer lTineiren eee is the government’s program 
manager for the ANGAS progran. LTCOL Lasche, USA, was 
interviewed by telephone. Ihe project s téchnolortcea 
OobJeEctives aoa ls , potential applications, program 
status and evaluation techniques used to analyze 
proposed contractor costs will be presented in the data 
SCCELOnwe nh. talemenha mire ie 

The government program manager for the ANGAS 
program authorized access to the principle investigator 
at the Lockheed Space Sciences Laboratory in Palo Abeae 
CA, iter George Nakano. Lockheed’ s princi pie 
investigator for the ANGAS project is responsible for 
technical supervision’ and overall project direction. 
Dr. Nakano was interviewed by telephone and visited at 
the Space sciences Laboratory. His description of the 
ANGAS cost estimation effort, accomplishment "or 
technological goals, spending status, problem areas and 
significant SOA extension achievements will be 
presented in the data section Of vthts chapter, Anpenw 
B ands Chapter. [Ve 

Cost estimation procedures Sused by Lockheed 
wie tie obtained from Lockheed’s Estimating Systems 


Description Manual and from a presentation on SOA 
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Piet oy es Mreeee Lea, Castro,._Manager. of _Lockheed’s 
Estimating Systems; Mr. Donald H. Palmby, Manager of 
Lockheed’s Cost Modeling and Analysis; and Mr. Ken 
Peeler, Chief Industrial Accountant of Lockheed’s 
ASeromautics Division. 

Pit Ota toOtimmanG Cchanaemeristics of bottom-up 


cost estimating, parametric cost models and Lockheed’s 


Poe ria Lay eve lowed STAR cost 6S hima lonmwewere 
Obrai ned. Pierre Par anetelIc sand SOutLput data were 
obtained for sample SOA extension component cost 


estimates using the parametric and STAR cost estimation 
models. ties int ormation was detailed in Chapter II. 
2. Data Analysis 

The estimated ANGAS development costs as 
provided in Lockheed’s contract proposal and modified 
aiter government audit and negotiation are compared to 
aemuiali IFeported costs. Differences and trends in 
Specific cost SCaveporiles are aime y Zed Overall 
SOemaing rates are compared with anticipated spending. 
Meemresults of the data analysis are presented in 
Chapter IV. 

Significant characteristics of Lockheed’s cost 
Siero Metneds and models that _are potentially 
useful in aiding better estimation of SOA extension 
Meetecctseoy the government are also recorded in Chapter 
ey. 

Tice specific =» metnodelogy that will be used in 
the cost analysis will closely follow Dearden’s simple 
Poe bamecmmcmalvysis as detailed in Cost and Budget 
Analysis. Actual costs will be obtained from the ANGAS 
"Weekly Financial Status Report" and compared with the 


Cstamatvegmor "budgeted costs™ in a variance matrix 
Forma &. 

Rua seted costs will demic mine — Dlanned 
expenditure for work scheduled for accomplishment 
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during a specific phase of the contract. The Deeeered 
cost from work scheduled (BCWS) for a level of effort 
CONntrace, like the ANGAS contract, "is based on 
expected expenditures by tasks to be completed." [REF. 
9s pee: | 

A determination of the degree of completion for 
stated technological goals and task completion is 
provided in Chapter IV. Information obtained froemm@ene 
government’s program manager and the ANGAS principle 
investigator about the status of the SOW items will be 
the basis for establishing task and technological 


progress. The reasons why the program manager was able 
to 
Beer ay os possible tee basic 
Techn Og cal So pe ves” which are of primary 
interest e152 54 


will also be discussed in Chapter IV. 


C. DATA 
ili Introductiom 
The Research and Development Division of LMSC, 
Palo Alto, CA, has been funded through an incremental 
cost-plus-fixed-fee contract to develop the ANGAS. The 
major milestones are presented in Appendix B. 


In a _cost-plus-fixed-fee contract, the government 
pays allowable cost and_a fixed fee. Fixed feemtiiece 
no vary with actual costs. Provides minimum 
incemeive 1Or COn era c Lows EO eomatrol costs. 
Completion requires COntrac tor to deliver end 
product. Requires .~specitved ~Teveleeot “eit fort tome 
Stated) period of ime. Used most widely. for 
iG 3 eon preliminary exploration or study. [RE 
-D. 


The following information details ANGAS funding to 
Giankie 


PERIOD Si COD DE pre COST PLUS Bre 
if : 2 Sl oS 68,345 2 1,350,000 
z 3,092 70120 192,642 3,585,463 
TOTALS > 4,674,278 S 2e lakes $ 4,935,463 
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LMSC submitted a contract proposal on November 
Zo l9aa5 aitersthe first incremental funding had been 
approved, that covered a Jemnonth period commencing 
wt eit 956 through completion. The government 
choose not to allocate funds for this entire 6 1/2 year 
Poamrocmaonas Nhasmimerementally fumded the project. The 
contractor’s proposed cost for completion of ANGAS was 
$16,589,398.00 (CPFF). Appendix B contains the SOW 
items from the two incrementally funded contracts and 


PiemMrumaed propasea comoletion contract. 


Palo COSTS FOR FIRST CONTRACT PHASE 
Pitt csecaren sand Development Division submitted 
the following breakdown of proposed costs for the 
imferal ANGAS contract: 
PRICE BREAKDOWN 


DeoCklPTLON TOTALS 
LABOR HOURS 14,688 
LABOR DOLLARS > 23197520 
OVERHEAD 5 Ooi 2 
eee EE APENSES 3 Geno 2 
MATERIALS jek Ole: facie: 
SUBCONTRACT 146,700 
PeeewREMENT BURDEN eo 1 2 
TRAVEL 16,064 

SUpeeecOs TS S12 tersas 
COST OF MONEY we , oa 

Oral COST leh 

FIXED Oe 7 

TOTAL CPF Dp poe liwlec 


The individual cost element totals were 
developed as follows: 


i. Perectsbanor 9§—- direct labor hours are based_ on 
a technical evaluation developed from detailed 
Mieevcis Ol the “(program requirements. — e 
iamererabLes sore cUltentmmaverages for direct 
labor pools, by skill categories. 
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HOURS RATE AMOUNT 
DESIGN ENGINEER 2,324 $18.31 ‘$42,546 
DESIGN ENGINEER - SENIOR a Oe eo 558,638 
DESTGN (SPE Grab is. 664 $26.13 $17,350 
ENGINEER (SObENT TEuC CLER LC Aiea D 179 ‘12.46 2a oO 
LABORATORY ANALYST - RESEARCH 1,162 $16:67 $193376 
RESEARCH ENGINEER 1,148 818.87 $21,661 
SCTENdiass 1 > SENIOR 1,323 « Bl7 . 89 (8237755 
SCIENTIST - RESEARCH 2, roe) ae $64,391 
STAFF ENGINEER 621 ‘'S2@8.07 17,430 
> TARR SCILENTISs L771 fae Oe $49}608 
LECHNICAL PUBL RGAE rans 179 $14.78 2,645 
TOTAL DIRECT LABOR 14,688 $21.76" S$ 3g os8 
ie Overhead rates are applied as 4 percent = ie - 
direct labor dollars. The rates used for 1985 
and 1986 were 1.592 and 1.560, respectively. 
These rates apply to R & D projects. 
BASE RATE AMOUNT 
OVERHEAD 1985 $2308 0 Sele oc 3760 55a 
1986 SS, 107 eel. DOU 129,647 
TOTAL OVERHEAD $319,630 $506,192 
ore General and Administrative Expenses are also 
a eee as a percent times direct _apen 
dollars. The rate used for 1985 and 1986 were 
-424 and .417, respectively. 
BASE RATE AMOUNT 
G & A EXPENSES 1985 Re aaa 0.424 ee 
1986 83,107 0.417 34,656 
TOTAL G & A EXPENSES $319,630 $134,942 
4, Material requirements were listed, identified 
and priced by component. Costs of purchased 
services eare also included in this geecoecr 
e Venere 
MATERIAL NOSo 51,547 
PURCHASED SERV. . 1985 59,246 
TOTAL MATERIALS $110,793 
Sc Subcontract cost estimates are based on actual 


price quotes received from the subcontractors 
in response to a RFP. Subcontract costs for 
1985 were estimated at $146,700. 


6. Procurement burden is applied at the rate of 
OO of the total estimated costs for 
Maver 1 287 eee services and subcontract 





GOS ES« e rate is developed from historical 
data. 
BASE RATE AMOUNT 
1985 257,498 0.035 “739,012 
TOTAL PROCUREMENT BURDEN $9,012 
Wee Travel - per diem rates are based on historical 


data and air fare costs are based on tourist 
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class rates. Mavelmmmet lects antietpated 
Pprojecterequirements. 


QTY RATE AMOUNT 





AIR FARE aa 
ALBUQUERQUE 1985 4 $ 193 Ye 
omer iro OF 2 MEN FOR 3 DAYS 
TECH MEETING 
DENVER 1985 Awe 325 861,300 
2 BRIPS OF 2 MEN FOR 3 DAYS 
INTEGRATION COORDINATION 
LOS ANGELES/ BURBANK 1985 a Sele 870 
So URIeS OF 2 MEN FOR 1 DAY 
FET. COORD. MEETING 
WASHINGTON/ BALTIMORE Oso) 2 bepolome 6, 6 / 2 
OU RMEPsSs OF 2 MEN FOR 3 DAYS 
VENDOR & CUST COORD. 
PER DIEM 
ONE DAY TRIPS 6 45 270 
WEert COAST TRIPS Ze See lo 4 
Exel COAST TRIPS 36 111 3,966 
TOW AI Wea Fae $ 16,064 
8. Cost of mone is an element of facilities 
Gapital. Rates are applied to direct labor 
dollars, general and overhead base and the 


procurement burden base. 


bg 
> 
op) 
J 
a 
> 
A 
J 
no 
Ss 
oO 
= 
at 
=a 


Py oor hin i= LMSC iS. ZaOmocs Oo 24,386 
1986 ao, LO Seo. te2cs3 PO 0S 

G & A EXPENSES ei eoomoes 0.01055 eee OD 
1986 Sop / Oma o4 moat 

PROCUREMENT BURD oso 257,493 0.00074 eg 
DOr COS! OF MONEY ects), Ce 


The total of the cost elements is $1,281,655. 
Lockheed submitted the contract proposal with a fixed 
fee of $99,467, 8 percent of costs. The total CPFF 
requested by Lockheed for the first phase of the 
Conmeract was $14381 ,122. 


E. DCAA AUDIT REVIEW 
Lockheed’s cost proposal was audited by DCAA, 


sunnyvale, CA. The results summarized below are set 
LOttimeiimaladit report 7481 — 5G 210091, dated 5 June 
19'S 3 : 


oe 


COST CONTRACTORS QUESTIONED 
ELEMENT PROPOSAL COSTS 
SUBCONTRACTS $ 146,700 $2537 9 
PURCHASED SERVICES 59,246 99,246 
MATERIAL 51,547 Slot 7 
PROCUREMENT BURDEN a012 4,766 

$ 266,505 $ 140,938 


DCAA questioned the subcontract costs of $ 
146,700 because a cost or price analysis had not been 
completed by EMSC. Cost or price analysis for 
subcontracts #91s ™ requi redieny, FAR 15.86. DCAA, 
therefore, applied a 17.3 percent negotiation reduction 
factor to the proposed costs. The 17.3 percent factor 
is based on DCAA evaluation of subcontract history in 
the Astronautics and Space Systems Division. 

The purchased services of $ 59,246 and material 
of $51,547 were questioned because LMSC did not provide 
any data in support of the proposed costs. 

The procurement burden was reduced due to the 
above questioned costs and the reduced procurement 
burden base. 


QUESTIONED BASE COSTS $136 eae 
PROPOSED RATE Xess 


TOTAL QUESTIONED 


$ 4,766 


J Summary Of Cost And Fee Negotiations 
LMSC submitted an SF 1411 with supporting 


documentation to justify the questioned subcontractor 
costs. This documentation was reviewed and approved by 
the negotiator and scilentili te vot ace 

LMSC also provided a detailed breakdown of the 
questioned costs for purchased services and materials 
Which was approved by the government. These price 
breakdowns were based on past experience on similar 
projects, such as project Winkler, catalog prices and 
vendor quotes. 

All questioned costs were reviewed and accepted 
after negotiation. LMSC had requested a fee of 8 
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percent of costs ($99,467), the government’s' fee 
objective was 4.2 percent. As a Be sult aot 
negotiations, ee settled fee was 5.5 percent 
($68,345). The final estimated costs for the first 
phase of the ANGAS project was: 
Pol bate D COST FIXED FEE MOB 

oles 1 5000 S$ 68,345 peo oo, 000 





ocr tee COSTS FOR SECOND CONTRACT PHASE 

LMSC submitted a cost proposal dated 26 November 
1985 in the amount of $16,589,398, covering a period of 
performance of six and one-half years. Audit report 
7481 - 6G210136 conducted by DCAA, Sunnyvale, CA, 
@@estioned portions of every item in the budget. LMSC 
was unable to substantiate its cost estimates to the 
satisfaction of the government. LMSC was unable to 
accurately project costs six years into the future for 
a program still in the design phase. 

It was decided to contract only for the final 
design and sensor shield evaluation of the ANGAS 
project. The SOW is illustrated in Appendix B. LMSC 
proposed a cost of $ 3,694,580. The costs were broken 
down into the same cost elements as the first contract 
detailed earlier in this chapter. 

1. DCAA Audit Review And Negotiation 

The following proposed costs were questioned 

and negotiated: 








i. Labor Rates - LMSC used an escalation factor of 
4.5 percent. Due to the government ceiling on 
salary escalation, LMSC was instructed to 
mecompute labor rates using a 3.5 percent 
escalation. 

CATEGORY PROPOSED NEGOTIATED TOTAL 
es RATE RATE HOURS 
Pet wea ie oeeCIALIST 24.66 24.42 HO 2A) 
TECHNICAL STAFF So Ulole 30.35 6,512 
TECHNICAL SENIOR 21.45 21.24 10,460 
TECHNICAL ENGINEER ee OL 172.74 ooo 
iecna eA SUPPORT 13.00 ea 4 7 Sire 
LABORATORY SERVICES once eel ll 2, 0016 
TECHNICAL PUBS. 15.94 a 2S 493 
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ae Fee Negotiation - LMSC requested a fee of 9 

pee eke of costs, exclusive of C08 t Oe 

he government’s fee objective was 4.9 percent. 

A fee of 5.8 ercent, exclusive of cost-of- 
money, was negotiated. 


au Summary Of Estimated Costs For 
The Second Contract Award 


The following enart provides the, f ina 
estimated costs for the second contract award after 
audit and negotiation: 

PRICE BREAKDOWN 


DESC Tea) TOTAL 
Labor Hours 40,052 
Labor Dollars $ 891,783 
Overhead i, oO ee Wizes 
G&A S61 le 
Material 583 ,caa00 
Purchased Services 6,000 
Subcontracts 102,96s 
Procurement. Bincen aa 5 wow 
imravel 3176528 

Subtotal $3,301, 55" 
Cost of Money Give 

Total Cost $3,392 , Ge 

Pee 192 ,ea 

Oe EF: $3,585,448 


G. ACTUAL COST DATA 

The level of funding, actual expenditure level and 
planned spending as of June 30, 1987 is illustrated 
graphically on the following page: 
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CeO oer Sele eel 6S 


As of June 30, 19867 


i Fanosive 


(Tnhousande) 


+ AACTUAL 


g CURRENT 
( LAN 


Dollare itn Millione 





3 09 NWN D J F M A M J J kb 8 O W DPD J FP MA M 
Month ending on mark 1987 -—— 4988 


Actual cost data were obtained from Mr. Don 
POesciu, LiaG S cOmtract administrator for the ANGAS 
project. The source of the following spending data is 
meme veeklys Project Financial Status Report, dated 
Semecinber 31, 1986, which reflects spending at the end 
or tm@e first contract phase. 


ACTUAES COSTS 
Kimo l CONTRACT PHASE 


Pew Deo. DATE Bewer T 

LABOR HOURS 17,424 hours Lonel) 
PABOR MCCsls S SOURoLS dD 302,464 
OVER AD S 556,858 § ORO oa 2 
Cee A BAPENSES $ oc Od 147,780 
Citi TL. PREMIUM S 24 §$ 0 
MATERIAL $ oy sa. Oe Oo 
PorcEAobBDSSERVICES $ 360,099 $$ 0 
SUL COrnnwsA CT. $ 0 6$ aor 7 U0 
PROCUREMENT BURDEN S Pore OE ble2 
TRAVEL S OOO nO 16,064 
GRAND TOTAL Sameer! (Omomo > 1,243,205 
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The difference between the amount funded under 
the contract and the budgeted total shown above is the 
cost of money and fixed fee amounts. 


TOTAL FUNDED ($1 350,000) = TOTAL BUDGETED ($1 243,205) 
+ COST OF MONEY ($38;450) + FIXED FEE ($68,345). 


Chapter IV discussés the actual “budsceren] 
accomplishment and measure of technological progress 
for the first and second phases of the contract. 

The following cumulative actual costs as of 
September 31, 1987, provide actual expenditures for the 
first and second contract phases. 


ACTUAL COST 
FIRST AND SECOND PHASES 


EXPENDED TO DATE BUDGET 
LABOR HOURS 95,585 hours 73,6053 
LABOR COSTS $1,170,014 $ 1,3697e67 
OVERHEAD $ 1,806,416 $ 2,023,831 
G & A EXPENSES $ 54253092". 5S 709,570 
OVERTIME PREMIUM $ 118 § OG 
MATERIAL $ SOOO 7 1 eas 19oRiZue 
PURCHASED SERVICES $ 59, 11s ol arue 
SUBCONTRACT $ 0 §$ 146,700 
PROCUREMENT BURDEN $ Loe AeVG- 3s 12,380 
TRAVEL $ 25,606 § 18,064 
OTHER . $ Some Laie 
GRAND TOTAL > 3,977,768 ~$)4,5337 750 


The difference between the amount funded junce. 
the first and second contract phases and the budgeted 
total shown above is the cost of money and fixed fee 
amounts. 

OLB br BN HBwhY! (3220883 7s TERED PRET EB of $4899 99°? 
Ho: INFORMATION FROM THE GOVERNMENT’S PROGRAM MANAGER 
The following discussion is based on a telephone 
conversation with LTCOL G. Lasche, USA, the ANGAS 
program manager on September “cys = LMSC initially 
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proposed the ANGAS project to the government. At the 
tiie wmoeKneea Had recently completed the four million 
dollar Winkler project. Winkler was a scaled down 
aircraft-borne spectrometer that utilized many of the 
advanced technologies that were to be used in the ANGAS 
Dio vec. 

Winkler was successfully developed, tested and 
flown at the estimated costs. LMSC’s proposed costs 
for ANGAS were evaluated using Winkler’s historical 
COS taGia ta . 


ANGAS’ technology was more advanced than 
Winkler’s because it is designed to function in space 
rather than the earth’s atmosphere. According to the 


program manager, projects like ANGAS that incrementally 
advance the SOA are usually completed much closer to 
the scheduled time and budgeted costs than projects 
that attempt quantum leaps. 

The incremental funding of the ANGAS contract 
allows a structured development and technological 
evaluation at key development steps. LICoL. Lasche, 
USA, stated the contract’s SOW items and required tests 
stipulated in each contract phase provide the primary 
evaluation criteria to measure technological progress. 
To date all SOW items and tests have been 
satisfactorily completed on time. 

It was also noted that labor costs are usually 
the most significant expense during the research and 


development phases of an SOA extension project. 
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IV. DATA ANALYSIS 


A. INTRODUCTION 
This Chapter analyzes the data presented in Chapter 


III. The analysis examines cost variances, schedule 
variances and oe measurement of technological 
progress. Significant characteristics of Lockheed’s 


cost estimating methods and models that are potentially 
useful in aiding better estimation of SOA extensions by 
eles government will also ie discussed. The 
contributions and role of the government’s program 
manager for the ANGAS will also be highlighted when 
applicable. 


B. COST VARIANCE ANALYSIS 

Table 2 provides the dollar value and the 
percentage of cost variances by cost element for the 
first funding period of the ANGAS project as of 31 
september 1986: 


TABLE 1 
COST VARIANCES 

COST ACTUAL BUDGETED COST VARIANCE AS 
ELEMENT COSTS COSTS VARIANCE % OF BUDGET 
LABOR $ 369,323 $ 302,464 $ -66,859 22.10 % 
OVERHEAD DOR SUS HOR S92 -46,466 9.10 
G&A 184,323 147,780 -36,543 24.73 
OVERTIME 24 0 = 24 NA 
MATERIAL Po, o33 T1LO% 7 93 +95 ,260 85.98 
PUR. SERV. 36,099 0 +36,099 NA 
SUBCONTRACT 0 146,700 +146,700 100 
PRO.BURD. 10s Sar + 7,334 S12. 36 
TRAVEL OF 00m, 16,064 + 9,407 Des eno 
TOTAL $1,170, 515 ae oleae +$72 ,690 5.8% 


A a 


Table 1 indicates that negative cost variances? 
during the first funding phase of the ANGAS contract 
were realized in labor costs, overhead, G & A expenses 
and overtime premiums. These cost elements Beare 
imme meocd Dy slaporecosts etther directly or through an 
applied rate to a direct labor base. 


LMSC had budgeted for 13,900 labor hours. ae 
number of labor hours actually expended during the 
LP uoseepnasewOrmune  “COntract was 17,424 hours. Dae 


George Nakano, LMSC’s Principle Investigator for the 
ANGAS program, explained that during the early stages 
of development additional research and development 
effort was put into the solid-cryogen cooler system. 
This was aocritical item in the overall SOA extension 
effort that has a long lead-time and would be built 
Mieataec. bnNe Space =selences Laboratory Division. 

mile Other cost elements experience positive cost 
Tal lanceemaunmime the initial funding. A cumulative 
positive cost variance of $72.690, 5.8 % of budgeted 
total costs was reported at the conclusion of the first 
Comueract period. 

Budgeted labor costs, overhead, G & A expenses and 
Sveruine premiums, all based on direct labor costs, 
were budgeted at 77.27 % of total budget costs. Actual 
costs for these cost elements accounted for 94.87 & of 
actual total costs. 

Table 2 provides the cumulative and percentage 
variances for the first and second contract phases as 
Saeco eeecpuemper 1987; 


4 Negative cost variances result when actual costs 
exceed budgeted costs. 
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TABLE #2 
CUMULATIVE COST VARIANCES 


COST ACTUAL BUDGETED Cost VARIANCE AS 
ELEMENT COSTS COSTS VARIANCE % OF BUDGET 
LABOR bl, 17080 14 ect +$199,853 14.59% 
OVERHEAD 1,806,416 $2,023,831 +4+217,415 Ur 
G & A 542,392 109, 0 20)9 +163 iS Zora 
OVER This 118 100 - 18 18 
MATERIAL 360,671 196,203 -164,468 S3anoe 
BUR aor Re. ao, 1 10 61,110 + 2,000 eee 
SUBCONT. 0 146,700 +146,700 100 
PROP DUR. IS. 6 12,380 - 736 oa 
ea ie 25,606 LSAwGss) — 7), sa 41.75 
OTHER aio 25 > = 200 160 
TOTAL $3, 9777 OS 55$4), 5330950 “Eoeo, Le2 Ie ercioe 


Table 2 indicates that negative cost variances 
existed at the end of the second funding phase in 
materials, procurement burden, travel Sand others cocet 
element catagories. A cumulative positive cost 
variance of $556,182, 12.263 percent of budgeted total 
costs, was reported at the conclusion of the first two 
contract periods. 

LMSC had budgeted for 73,653 labor hours at this 
state of the development effort. The cumulative number 
of labor hours actually expended during the first two 
contract phases was 55,585 hours. 

Direct labor and direct labor-based costs were 
budgeted at 90.4 % of total budgeted costs. Actual 
labor-related costs were 88.4 @ of actual total costs. 

Total actual costs for the ANGAS project have been 
slightly less ethan ~pudzeved! Dr. Nakano and Mr. 
Dorsett, of LMSC, explained that the slightly slower 
overall spending rate was partially attributed to the 
uncertainties and delays fexper lemece in *@b cain ine 
timely approval of incremental funding requests. 
Material procurement and research efforts that are 
needed for SOW items defined in future phases of the 
project would have been initiated in the first and 
second phases of the project if future funding had been 
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apne vedwe the Imeremental funding Approach adopted by 
ico OC aiMemt | DRevIdes controwmeover the project but 
also makes the development more difficult for the 
COmMurae Lor. 

Incentives and definite evaluation periods for a 
fore home nmecoltractw likemetm@e ANGAS project are 
Sop ourmes ied through ce ime treme tira. Punad m2); 
eouamretment, Ol ~sO0Wweo9Oals LTorseach contract phase and 
it otmrt Ey oe tO Ad )jast  LUubkunmes funding based on an 
evaluation of past performance and future potential. 

Titec OuwumaiclOMmmlac sWeen meprovided incentive to 
See aerorily achieve stated SOW goals for the phase 
of the contract for which funding has been reviewed. 

tiewere long lead-time items required in later 
contract phases On ae ie ANGAS project cannot be 
Vitieeared ambit ttnding of those contract phases is 
approved. 


Ce oCOE MOLE VARIANCE AND TECHNOLOGICAL PROGRESS 
Seleqdure me ameeaices S DrOViIidemmawsway to highlight 
Beat lOwsewiere elbrort is behind or ahead of schedule. 
The traditional methodology for analyzing schedule 
Variances compares the budgeted cost of work performed 
ana tne budgeted cost of work scheduled. tae. lod 
Sinem CeveropiemL COnUracte, suen ac ime ANGAS project, 
a determination of the budgeted cost of work performed 
becomes almost impossible to calculate due to the 
Pant tiees nature of “the goals =and SOW items. An 
Svssatron OL. teemnical  pregress Sis a more useful 
measure when used with the project’s cost variances to 
evaluate overall benefits and project status. 
Mico eomOot oLeCiunical Vartance mts probably the most 
eee see soneutwieres sasistance: iof otner 
technical experts may be required. It may_also 


heQdw@@rew-some restructuring of the program merely to 
determine the cause of the variance. Note that any 
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significant technical variance will always result in 
cost and schedule variances. {LREF. 2:p. 4230] 


Since no significant cost or schedule variances 
have occurred in the ANGAS project, the most obvious 
indicators of unsatisfactory technical progress are not 
evident. LMSC has satisfactorily completed all SOW 
items listed in the first and second phases of the 
contract. To further amalyze technical \prer, ec. ine 
ANGAS project’s system engineering will be examined. 

MIL-STD 449 defines system engineering as: 

the application of scientific and engineering efforts 
to: transform an operational need into a description 
of _system performance parameters and a system 
SE Geta through the use of an incentive process 
of definitions, synthesis, analysis, design, test and 
evaluation; integrate related technical parameters 
and assure compatibility of all physical, functional 
and program interfaces in a manner which optimizes 
the total system definition and design,; integrate 
reliability, maintainability safety, survivability 

human an other such “factors -into the tota 

engineering effort. 

The contract SOW specifies the minimum technical 
requirements 12 > tieweeconvtwacec. Appendix B contains 
copies of the first and second contract phase 
statements of Work. Examination of these 
specifications reveals that they are composed of 
Spee it lec configuration, design, PEerLormance and 
reliability and maintainability requirements. ine 
program manager, LTCOL G. Lasche, USA, stressed that 
the lessons learned in- the development of the Winkler 
project, which used similar technology, was a basis for 
the technical requirements developed for ANGAS. ANGAS 
system engineering appears to be a detailed 
specification of valid technical goals developed from 
past experience and an overall knowledge of the 
technologies involved. 

It is logical that any analysis of technological 


progress be based on status reviews of system 
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esiMecramo goals. lie government’s acquisition 


process requires formal and informal design reviews to: 


G@ecermimme the adequacy of ~CGontractor and DOD in- 


house 
EarGtlelpants should include 


EErorts towards aes Pee cae Panes 
esign a eaeiedein ae 


spe@eialists in BO en lg maintainability, safety, 
Ce 


and, 


eritdeal design Preview 


Par ticibar ly. bogist 


SMpoOhtaviveity. Reviews 


should _include a ew" (CDR), a Gesign review a 


Lev le Ween DUNC LIOnail con 
configuration audit, and a 
ComlLpuratlomeinspections [REEY 2:p. 4-4 


CDR a design certification 
Poupamuon, “audit, a physical 
Ame ere 


The most recent formal review of the ANGAS project 
was the CDR which was presented to DARPA on July 21 and 
22 “On 1982. 


Tew, a toOrmal review of the detailed design of a 
cont reuration ee i eperiormea by the preceam 
e W 


manager late in prototype subphase Cra ie 
desten wWdetai il is essentially complete, but prior to 
eR TREE. and fabrication of Lorna iia ves t 
articlés. LREF. 2@:p. 4-49 


The 


Oral t1-62 1 design review demonstrated that 


technical and system engineering goals for the ANGAS 


project 


were accomplished satisfactorily. The CDR 


specifically addressed: 


ie 


Ox 
flisties 
Nee 


la 


System Engineering 

Germanium Detector Subsystem Development 2 
CWweereal «tecnica l component of the ANGAS 
system) 


shield on (a critical technical component 
of the ANGAS system 


Mechanical Engineering specifications 
Electrical Subsystem specifications 
source Deployment Subsystems 


Cooler Subsystem specifications ( 


a nti ca): 
technical component of the ANGAS een 


Erectrical Ground Support Equipment 
specifications 


Monitor Subsystem specifications 
ns tLrument slest ~Pilan 
Reliability, Quality and Safety 
Instrument Development 


summary, ANGAS technical goals and objectives 


have been accurately detailed and specified as contract 
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requirements based on past experience obtained from the 
development of systems using similar technologies. 
Formal and informal reviews and evaluations of 
technical requirements have proved that the ANGAS 
project’s technical goals are both viable and capable 
of being accomplished. 

This research effort does not attempt to define a 
Gian et ba tive measurement technique for evaluating 
technical variance and accomplishment. An 
understanding of the long-range R & D planning steps 
developed by Ward D. Lowe and presented in Chapter III 
is a useful guide in establishing initial technical 
objectives for a proposed SOA extension project. 

The prior development of the Winkler Program 
reduced the risks and development and provided 
technical information on some of the new technologies 
used in ANGAS. The following brief description of the 
revabkaivie ly) —Sdinwoace multi-attribute Utility Method: 
summarized for the Navy Program Manager’s Guide, 
provides an avenue for future study of quantification 
of technical goals and measurement. 


The model addresses the management of risk that is 


inherent in decision making with incomplete 
information. Establishing SOA extension technological 
objectives and measurement specifications at the 


initiation of a project is often based on incomplete 
information. The use of this limited information can 
be optimized through a five step process which consists 
Ghia, 


i. Pe a eee down the tasks to be accomplished into 
manageable components or attributes. 


ae Estimating the utility t42crte the Petar a 
importance of each component or attribute. 


Se Developing a a rl function or curve whiten 
describes the utility values as a function of 
some descriptive variable (i.e., pe) eee in 
terms of mean time between failure. 


one) 


4. Ecsuimatingetne risks associated with attaining 
the utility values chosen for each attribute. 


D. Developing eptions to avoid OP Overcome 
Shbstacles@re,suceesseamasueo, cCOmpare alternative 
patis, SOtUuElons, Orw=contepts. 


Dey SIGNESICANT CHARACTERISTICS Die OChmHeE Ds) COST 
ESTIMATING METHODS AND MODELS THAT CAN AID 
GOVERNMENT COST ESTIMATING 


Lockheed sometimes uses bottom-up cost estimating 
which is usually based on historical data and provides 
aes high degree of accuracy. Cost estimating and 
Voluiai tone wileain DOD should utilize histort@al costs, 
Lore conmpletedue proj fects@enat ~are Similar in scale and 
Eeemmolosy, to evaluate projects under consideration. 

Use oOf@parametric cost models, specifically the RCA 
PRICE Hardware model is used by Lockheed during the 
early development and concept exploration phases when 
ba storical cost datagdoes not exist. Patame trie cos t 
models, which allows economic SOA extension trade-off 
and schedule analysis, Should be used by DOD when 
Dist Ormco. data sesadees snot exist early in development 
stages. 

tne SLlockheed Vinternally developed STAR model 
combines the characteristics of a similar-to-model and 
adeassoetated models This model, which was described 
Piece tall tn Chapter 114 is based on a historical 
database that has been developed over a 20 year period. 
tae data base hepresentcommactual production and 
agevelopment cost experience, which can be retrieved by 
mince wiacecmmor pox level based on relatively simple input 
Cmaraceertstics. Development of similar integrated 
data base and models within DOD or major components 
Mould swemificantly Enhance the government’s ability to 


accurately estimate SOA extension costs. 
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V. CONCLUSIONS 


The conclusions drawn from this research effort are 
based on the data and analysis of the ANGAS project. 
These conclusions can be used as a basis for future 
studies involving SOA extension cost estimating methods 
and case studies. 


Primary Research Question: How do estimated 
development and production costs compare with 


actual costs for the ANGAS project? 

Development costs were the primary costs estimated 
and recorded to date due to the early stage of 
development of the ANGAS project. Total actual costs 
were slightly lower than total budgeted costs. Direct 
labor and labor related costs were higher’ than 
budgeted during the West phase of development. 
Variances did exist within individual cost elements. No 
cost element experienced negative cost variances during 
both contract phases that were evaluated with the 
exception of a $24 and $18 cost overrun in overtime 
premiums. 

subsidiary Research Questions: 

1. What methods and techniques were used to estimate 
development costs? 

LMSC used a bottom-up, engineering cost estimating 
methodology to estimate development costs. These 
estimates were generated within the Lockheed Space 
Sciences Laboratory based on past experience of costs 
incurred developing similar satellite borne systems. 
The bottom-up estimating methodology used at LMSC 
provided detailed functional and cost element estimates 
based on historical cost data and specific contractual 
requirements as detailed in the WBS, SOW and RFP. 


a2 


2. How was technological progress measured _ and 
evaluated? 

Technological progress was measured and evaluated 
by monitoring completion of the specific configuration, 
design, performance and reliability and maintainability 
Feqguirements Stated edn the ANGAS Statements of Work 
G@uring the formal and informal project reviews. The 
program manager for ANGAS was able to state technical 
goals and objectives based on experience gained from 
the development of a similar spectrometer system that 
used many of the same advanced technologies. 

3. What factors helped or hindered accurate cost 
estimating and accomplishment of stated SOA 
extension goals? 

Accurate cost estimating was enhanced by the 
Ememcucmece Of historical sdata on similar. type SOA 
SecctistOouyaevelopment costs. The division sof LMSC 
that was developing the ANGAS project was also 
gogoer enced fer developing similar systems. ANGAS 
mpierementaliy extended the SOA based on a smaller scale 
project that combined many of the same technologies but 
was designed to function in the earth’s atmosphere 
instead of outer space. Historical cost data, technical 
experience and incremental SOA extension enhanced cost 
estimating accuracy and accomplishment of technological 
goals. 

naerorm= —CNdAtshinme@er accurate cost estimating and 
accomplishment of stated SOA extension goals arise from 
the inherent risks of decision making with incomplete 
imtorpmation. The inability for the government to 
accurately specify project design and specification 
requirements and monitor accomplishment through formal 
and informal reviews greatly hinders accomplishment and 


measurement of stated SOA extension goals. Lack of 


o3 


historical data, technical” e€xpertrence lame iia, ae 
correculy interpret contract requirements hinders 
contractors from correctly estimating SOA extension 
COSts: 

Find Wcer wea were were no significant variances 
between total estimated and total actual costs to date. 
Due to the contractor uncertainties associated with the 
incremental funding used in the project the spending 
rate was slightly lower than planned. 

Incremental funding of a DOD SOA extension project 
definite contract phases and the opportunity to specify 
and evaluate technological objectives during each 
phase. This type of funding provides incentives for the 
contractor to SUCCES S HUMELY achieve technological 
objectives to receive funding for later phases. 
Incremental funding provides leverage and flexibility 
COE ae government in reducing cost overruns and 
achieving technological goals. The length of time 
required for the development effort is increased with 
incremental funding. 

Direct labor costs~)~ and costs applied as a 
percentage of direct labor costs were the most 
significant costs during the déevelopmen® piases. Piece 
costs accounted for over 75 percent of total costs in 
each phase. 

SOA extensions that incrementally advance the SOA 
reduce the risk of costs overruns and failure to 
achieve technological goals. The historical cost data 
and expertise obtained through the development of the 
smaller scale Winkler DPEOVECE enabled DARPA to 
accurately detail system engineering specifications and 
technical test parameters. 

The measurement and analysis of cost variances and 


technological progress is the primary gauge of 
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effectiveness for a research and development effort 
AO ied Dy Soemelevel on setrort contract.Technical 
progress for the ANGAS project was evalua leds as 
satisfactory. 

purrent ly available parametric, analogous) and 
Mie atOomeasu CaLimating models “useds by LMSC and 
PRevaAveminausitry = could be “adopted to a greater degree 
by the DOD to enhance SOA cost estimating accuracy. 

PCcolmenaattons bor burther study: Additional case 
studies are needed to identify trends in the cost 
estimating techniques and accuracy of SOA extension 
SlTiomec es I aewCOMDULCE Dased COSt estimating models that 
are discussed in this research effort and many other 
computer based models not addressed in this project 
merit a great deal of additional study. Identifying 
COst estimating methods that will allow the most 


fener Ccilenu wmce “Of limited DOD funding by cost 
Sime Ctl wed Vi Hed Creel new technologies aLeTiT SOA 
extensions and new system Gqdevelopmenti are vital 


national objectives. 
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APPENDIX A 
DEFINITIONS 


Bottom-Up Cost Estimation Model: A Cost estimating 
procedure based on detailed functional and cost 
element estimates prepared at_the lowest practical 
level of task and design detail. 


Budget: Estimated costs of the effort defined in the 
work packages and level of effort SOW items. These 
costs should be broken out at the_ cost account 
level. The estimated cost of the effort scheduled 
to be accomplished at a _ point in time within DQD is 
termed "Budgeted Cost of Work Scheduled" (BCWS). 


Critical Design Review (CDR): A formal review of the 
detailed design of a configuration item is 
performed by the program manager late in the 
prototype subphase when the esign detail is 
essentially complete but prior to drawing release 
and fabrication of formal test articles. 


Development Costs: Costs associated with research, 
planning and design for a new product or process or 
for a significant improvement to an eS 
product or process. Includes the conceptua 
formulation, design, and ete tenet O product 
alternatives and construction of prototypes. 


Intra-Lockheed Work Transfer (IWT) Items: Material 
items that are made at Lockheed and transferred at 
cost to another entity within the company. 
Includes a separate breakdown by cost. 


Parametric Modeling: Cost estimating procedure which 
utilizes a model comprised of algorithms and 
mathematic equations. Estimated costs are derived 
from De ee of cost to physical or 
performance characteristics. 


Purchased Items: -Includes material items not 
considered standard commercial items or raw 
materials. 


Raw Materials: Materials in a form or state that 
require further processing. 


Standard Commercial Items: Consists of items_that the 
contractor normally fabricates, in whole or in 
part, and that are generally stocked inventory. 


State of the Art (SOA) Extensions: Used in this thesis 
to describe a new weapon system or component that 
has incorporated an advancement in technology ina 
unique process or for a unique application. 
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1h each WBS 


Woo eer onary ¢ Thee WEs Gwettomary defines 
ea)/funetional task 


elfemembta and Gescripes the. tecnnt 
CON@Memt and responsibil ierece 


Work Break Down Structure: <A _  product-oriented family 
tree composed of hardware, Services and data which 
BectunEs I romeorea)] ect Se oe ELrOGigeuring the 
fe MerovleWmummameG — DrOductloOn er a defense material 
Pee mand Wat chmmecOMDleuelry sceriness the project 
program. 
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APPENDIX B 


STATEMENT OF WORK 
FIRST CONTRACT 


1. The work and services to be performed hereunder 
shall be subject to. the requirements and sstandarae 
contained in Exhibit A and the following paragraph(s). 


ars The Contractor shall carry out a program of 
research in advanced Say demectors — for "use an 
space which shall include the following tasks: 
es Conduct gamma-ray imaging studies, 
2. Establish instrument GSE requirements, 
S Conduct preliminary ee of germanium sensor 
and anticoincidence shield, 


4. Prepare digital SF eee processing system 
requirements and specifications, 


Ons Prepare cryogenic cooling system 
specifications, 


6. Conduct preliminary design of collimator, 
wheel, onboard hardware ermanium sensor 
erecironice and antishiel housing and 
electronics, 


ve Pe paige germanium sensor procurement 
specifications and antishield crystal 
procurement specifications, 


Se Complete final cooler design and performance 
simulation. 
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STATEMENT OF WORK 
SECOND CONTRACT PHASE 


Under Modification P00001, the Contractor shall 
conduct the final design and sensor/shield evaluation 
of a space demonstration experiment known as the DARPA- 
ee eminem mabeal ume iree-flyer satellite 
mission. The work shall include the following tasks: 


(1) Finalize design of a high sensitivity, 
foiee ener yp resolution imaging spectrometer, Known as 
tie aNGASSinstrument, background monitors and auxiliary 
equipment. 


— Perform laboratory evaluations and bench 
es Se Of he Ge sensor design and of the Nal shield 
system. 


a) Comawe & costar Stuales, Somptncer 
Simulations and HON aie aloes Cenc bests LOnmperiLormance 
VeCliMicabEom, Ol -=the imag ng PeCcini gues (One emp loved 
Mite maniAs ang tihnalize the aqesign of the collimator 
mask system for the flight instrument. 


. EP. Provide. the electrical, thermal, and 
env rronmental Specinieations and Ope (ibe nace 
information Hecesswy a lOmmlmbeorate ithe  DARPA-201 
bavwead aboard the satellite. 

4 5n) Provide all physical procedural, 
rune tional, and safety data necessary to generate an 


Experiment Requirements Document (ERD) consisting of: 


te} Interface Control Document. 


Ground Operations Requirements 
Document. 

Goof light Operations Requirements 
Document. 


(6) Provide the ee necessary to meet the 
identified DARPA-201 milestone schedules. Departures 
from these schedules must be mutually agreed to by 
DARPA and the LPARL. 


Ore Maintain an experiment quality program 
during the design and development of the payload. 


(8) Design and ind tia e development of the 
ground SU perat Sore ae GSE) necessary to test and 
ae dee e DARPA-201 payloa with the free-flyer 
satellite. 


Provide DARPA-201 progress’) and status 


9 ) 
information and support the mission working group and 
status reviews as necessary. 


og 


ATEMEN R 

16.48) I O 
PLE Pao NTRACT 
The Lockheed Missiles and Space Company will 


provide to the Defense Advanced Research Agency and the 


oy T OF WORK 
PROPOSED MILLION DOLLAR 
COM N CONTRAC 


Office of Naval Research the necessary personnel, 
equipment facilities and Services™ tO  COntmeeoeoDace 
demonstration experiment Known as’ the DARPA-201 ANGAS 
Experiment aboard a free-i lyere= satellite misstom faa 
the conduct of this work the following tasks will be 
performed: 


1) Design, develop, test and calibrate a high 
sensitivity, fine enerey resolution ~ macim 
spectrometer, Known as the ANGAS instrument an 


an auxiliary background monitor. 


2) Conduct design studies, computer simulations 
and bench tests for performance evaluation of 
the imaging techniques for intended employment 
on the ANGAS collimator mask wheel. 


3) Based on the timely receipt of the electrical, 
thermal and environmental specifications and 
other interface information from the 
Government provide. the necessary information 
to integrate the DARPA-20O1 payload aboard the 
sate e. 


4 ) Provide all physical, procedural, functional 
and safety data neces ary to generate an Experiment 
Requirements Document (ERD) consistent with the vehicle 
and interface requirements specified in the SIP From 
7a (1985). The ERD consists of 


a) Interface Control Document 
b) Ground Operations Requirements Document 
c) Flight Operations Requirements Document 


7 Provide the inputs necessary to meet the 
identified DARPA—201 “matestone scheculece 


6) Flight qualify the DARPA-201 payload through 
the maintenance of an experiment quality program during 
the design and development of the payload. te 
Principal Investigator will = have full YTresponsie1 iit, 
for conducting all aspects of the quality program. ~ihe 
quality program practices with the ONR. 


7) Provide the ground SO equipment (GSE 
necessary to test and integrate the DARPA-201 payloa 
with the tree-f lyer ‘save iiree. 


8 ) Provide logistics and operation See pae to the 
DARPA-201 payload during definition, integration , test 
and on-orbit operations. 


9 Provide DARPA-201 progress and status 
information using "the  Loeckiheed R & D standard 
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fateatememt — and accounting Bee e S SUP pO tt — hae 
mission working group and status reviews as necessary. 


10) Based on the timel receipt of government- 
fiend Sed computer-compatible acency data tapes, 
analyze the first two years of on-orbit data. 

11) Provide flight performance reports as required. 


12) Submit the scientific results acquired with the 
DAREa—2Ol instruments Bfor “publication in the open 
technical literature. 
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MAJOR MILESTONES FOR THE 
DARPA-201 ANGAS EXPERIMENT 


Pronece 
Milestones 1985 1986 1987 1988 1989 1990 1991 1992 


ICD Signoff-------------- 


Ground Opera @hO@ lie 
required documentation 


Flight,.0Opeta tion. —— = 
required documentation 


Hardware Development--------------- 
Instrument Test-------------------- 

Hardware Delivery------------------ 
Spacecraft Prelaunch Support------------ 
Launch-------------------------- ------------ 


Post Launch Operations--------------------------- 
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